1. Field of the Invention
The present invention relates to a motion detecting apparatus, a motion detecting method, and a storage medium storing a motion detecting program for detecting motion of objects using motion vectors obtained by a MPEG video encoder.
2. Description of the Prior Art
With extension of business hours in stores and increase in the number of unattended facilities, the demand for reinforcing crime prevention measures in such stores and facilities is growing these days. As a representative measure, one or more surveillance cameras are placed in a store so that the security personnel can monitor images taken by the cameras.
However, this method is problematic as it requires the security personnel to constantly watch the surveillance cameras long hours. To lessen the workload for the security personnel, a conventional motion detecting apparatus is used for detecting motion of objects such as persons in images taken by a surveillance camera. When motion is detected on the camera, the security personnel is alerted by an alarm attached to the motion detecting apparatus and watches pertinent images, with it being possible to reduce the load of constantly watching the camera.
This conventional motion detecting apparatus uses a background difference method to detect motion of an object.
The background difference method compares a present frame taken by a surveillance camera with a frame preceding the present frame and detects motion when the two frames differ. To be more specific, the motion detecting apparatus calculates a difference of luminance between the two frames in units of pixels and, when the calculated difference is larger than a threshold value, judges the difference as motion. The threshold value is used here to prevent small picture jitter or solar irradiation change from being detected as motion.
On the other hand, a MPEG (Moving Picture Experts Group) technique for coding images using a MPEG method is becoming increasingly common in recent years. In this technique motion vectors are calculated during image coding. A motion detecting apparatus that detects motion using these motion vectors is also conventionally used.
A motion vector shows the spatial correlation between a target block and a prediction block. Here, one block is composed of 16xc3x9716 or 8xc3x978 pixels. The target block is a block in a frame (target frame) which is to be coded. The prediction block is a block selected from another frame which is temporally close to the target frame. To select the prediction block, the other frame or a predetermined search area in the other frame is searched for a block closely analogous to the target block. In the case of static images, a magnitude of a motion vector of the target block is 0, since a position of the target block in the target frame is identical to a position of the prediction block in the other frame. In the case of moving images, the target block and the prediction block often show one part of movement of the same object. Accordingly, a motion vector of the target block shows the magnitude and direction of the movement of the object. In brief, the motion detecting apparatus that uses motion vectors detects motion if a motion vector of a block has a certain magnitude. More specifically, the motion detecting apparatus checks whether the motion vector of the block has a magnitude larger than a threshold value which has been set in consideration of image jitter or the like, and detects motion if the magnitude is larger than the threshold value.
In motion detection techniques it is desirable to detect only movements of objects. However, when luminance greatly changes due to strong flicker of a fluorescent lamp or light incident from a motor vehicle during night, a flat area which actually does not show an object in motion may be detected in the above conventional background difference method and motion detecting method using motion vectors. The flat area referred to here is an area, such as a wall or a floor, where luminance is relatively even in a frame.
In view of the above problem, the present invention aims to provide a motion detecting apparatus, a motion detecting method, and a storage medium storing a motion detecting program that can prevent strong flicker or light flash in a flat area from being detected as motion.
The stated object can be fulfilled by a motion detecting apparatus, connected to a video encoder which sequentially compression-codes images inputted from a camera, for detecting motion of objects in the images, the motion detecting apparatus including: an acquiring unit for acquiring a plurality of motion vectors calculated by the video encoder respectively for a plurality of blocks and storing the plurality of motion vectors of the respective plurality of blocks in a memory; a first judging unit for detecting motion vectors whose magnitudes are larger than a threshold value among the plurality of motion vectors in the memory and judging blocks corresponding to the detected motion vectors as possible motion blocks; and a second judging unit for judging, when a motion vector of a present block and motion vectors of blocks surrounding the present block have non-random directions and non-random magnitudes, the present block as a motion block, wherein the present block is selected from the possible motion blocks one by one, and wherein the motion block shows a part of an object in motion.
With this construction, when luminance greatly changes in a fairly short time due to light flash or strong flicker, the mistake of detecting a flat area (such as a wall or a floor where luminance is relatively even) can be avoided.
Assume an image is classified under an image area that shows an object in motion (motion area) and an image area that does not show an object in motion, the latter being further classified under an image area that shows a flat part (flat area) and an image area that shows a nonflat part (nonflat area). In the presence of light flash or strong flicker, magnitudes and directions of motion vectors are even in the motion area, magnitudes and directions of motion vectors are random in the flat area, and magnitudes of motion vectors are 0 in the nonflat area.
With conventional motion detecting apparatus that detects motion whenever magnitudes of motion vectors are large, not only the motion area but also the flat area is mistakenly detected under the above circumstances.
On the other hand, the motion detecting apparatus of the present invention detects motion in consideration of not only magnitudes of motion vectors but also uniformity of the motion vectors in direction and magnitude, so that the flat area which does not show an object in motion will not be mistakenly detected.
Also, since the acquiring unit in the motion detecting apparatus acquires motion vectors from the video encoder, the motion detecting apparatus itself does not need to calculate these motion vectors. Accordingly, it is unnecessary to equip the motion detecting apparatus with a dedicated motion vector calculating device, thereby realizing the motion detection with a simple configuration using a conventional video encoder.
Here, the motion detecting apparatus may further include: an area storing unit for storing a critical area set in a frame; an area judging unit for judging whether at least one of motion blocks found by the second judging unit is present within the critical area; and an alarm generating unit for generating an alarm depending on a judgement result of the area judging unit.
With this construction, only when the area judging unit judges that a motion block exists within the critical area, the security personnel is alerted by the alarm generating unit to watch images in question, so that the load of constantly watching images can be reduced. Further, since incorrect detection of a flat area caused by light flash or strong flicker is suppressed as a result of the judgement by the second judging unit. Accordingly, an alarm will not be mistakenly generated by incorrect detection and so the workload for the security personnel can further be reduced.
Here, the area storing unit may further store a critical direction corresponding to the critical area, wherein the area judging unit judges, when the motion blocks are present within the critical area, whether an average direction of motion vectors of the motion blocks is roughly similar to the critical direction.
With this construction, an alarm is generated only when there is movement in the critical direction within the critical area. Accordingly, it is possible to design such a motion detection apparatus that generates an alarm when, for instance, an intrusion through an entrance into a store or an approach toward a cash box is detected. Thus, the workload for the security personnel can further be reduced.
The above object can also be fulfilled by a motion detecting apparatus, connected to a video encoder which sequentially compression-codes images inputted from a camera, for detecting motion of objects in the images, the motion detecting apparatus including: an acquiring unit for acquiring a motion vector calculated by the video encoder for each of a plurality of blocks from the video encoder together with a first difference sum-total and a second difference sum-total of the block, and storing the motion vector, the first difference sum-total, and the second difference sum-total of the block in a memory, the first difference sum-total, expressed as AE0, being a sum-total of differences in luminance values of corresponding pixels between the block and a rectangle area which is present in a reference frame and corresponds to the block, and the second difference sum-total, expressed as AE, being a sum-total of differences in luminance values of corresponding pixels between the block and a rectangle area which is present in the reference frame and is directed to by the motion vector of the block; a first judging unit for judging blocks, among the plurality of blocks, which each satisfy AE0xe2x88x92AE greater than xcex1, as possible motion blocks, xcex1 being a threshold value; and a second judging unit for judging, when a motion vector of a present block and motion vectors of blocks surrounding the present block have non-random directions and non-random magnitudes, the present block as a motion block, wherein the present block is selected from the possible motion blocks one by one, and wherein the motion block shows a part of an object in motion.
With this construction, the first judging unit judges a block as a possible motion block when a difference between a first difference sum-total and a second difference sum-total of the block is larger than a threshold value. Accordingly, it is possible to prevent incorrect detection of a block in a flat area (where luminance is even) in the presence of small flicker.
When an image is classified as a motion area, a flat area, or a nonflat area, motion vectors with substantial magnitudes are detected in the motion area and the flat area in the presence of small flicker. With the conventional motion detecting apparatus that focuses on magnitudes of motion vectors, such a flat area is mistakenly detected as motion of an object.
In contrast, the motion detecting apparatus of the present uses the property that, while a difference between a zero difference sum-total and a minimum difference sum-total is small in the flat area and the nonflat area, the difference is large in the motion area. That is, the first judging unit detects motion depending on whether the difference is larger than the threshold value, thereby preventing the flat area from being detected by mistake.
In addition, the second judging unit judges the present block as a motion block when a direction and magnitude of a motion vector of the present block are roughly similar to motion vectors of blocks surrounding the present block. Accordingly, it is possible to avoid the mistake of detecting the flat area under circumstances where luminance greatly changes due to light flash or strong flicker.
Here, the motion detecting apparatus may further include a third judging unit for rejudging, when the present block judged by the second judging unit as the motion block satisfies |AE0xe2x88x92AE|/(|MVx|+|MVy|+1)xe2x89xa7xcex2, the present block as the motion block, where MVx is a horizontal component of the motion vector of the present block, MVy is a vertical component of the motion vector of the present block, and xcex2 is a threshold value.
According to the above property that the difference between the zero difference sum-total and the minimum difference sum-total is small in the flat area, the third judging unit judges the present block as a motion block only when the above formula is satisfied. As a result, incorrect detection is further avoided.
Here, the motion detecting apparatus may further include a fourth judging unit for rejudging, when the blocks surrounding the present block rejudged by the third judging unit as the motion block are not motion blocks and the present block satisfies |MVx|+|MVy|xe2x89xa6xcex3, the present block as the motion block, xcex3 being a threshold value, wherein the fourth judging unit also rejudges the present block as the motion block when at least one of the blocks surrounding the present block is a motion block.
It is assumed that an image area showing a relatively large object (such as a person) in motion is made up of a plurality of motion blocks. In other words, a block whose surrounding blocks are not motion blocks and whose motion vector has a large magnitude is not a part of an image area showing an object in motion. Accordingly, the fourth judging unit rejudges the present block that satisfies the above formula as a motion block, to further prevent incorrect motion detection.
The above object can also be fulfilled by a computer-readable storage medium storing a motion detecting program for detecting, through use of motion vectors outputted from a video encoder which sequentially compression-codes images inputted from a camera, motion of objects in the images, the motion detecting program including: an acquiring step for acquiring a plurality of motion vectors calculated by the video encoder respectively for a plurality of blocks and storing the plurality of motion vectors of the respective plurality of blocks in a memory; a first judging step for detecting motion vectors whose magnitudes are larger than a threshold value among the plurality of motion vectors in the memory and judging blocks corresponding to the detected motion vectors as possible motion blocks; and a second judging step for judging, when a motion vector of a present block and motion vectors of blocks surrounding the present block have non-random directions and non-random magnitudes, the present block as a motion block, wherein the present block is selected from the possible motion blocks one by one, and wherein the motion block shows a part of an object in motion.
With the above procedure, when luminance greatly changes in a fairly short time due to light flash or strong flicker, the mistake of detecting a flat area (such as a wall or a floor where luminance is relatively even) can be avoided.
The above object can also be fulfilled by a computer-readable storage medium storing a motion detecting program for detecting, through use of motion vectors outputted from a video encoder which sequentially compression-codes images inputted from a camera, motion of objects in the images, the motion detecting program including: an acquiring step for acquiring a motion vector calculated by the video encoder for each of a plurality of blocks from the video encoder together with a first difference sum-total and a second difference sum-total of the block, and storing the motion vector, the first difference sum-total, and the second difference sum-total of the block in a memory, the first difference sum-total, expressed as AE0, being a sum-total of differences in luminance values of corresponding pixels between the block and a rectangle area which is present in a reference frame and corresponds to the block, and the second difference sum-total, expressed as AE, being a sum-total of differences in luminance values of corresponding pixels between the block and a rectangle area which is present in the reference frame and is directed to by the motion vector of the block; a first judging step for judging blocks, among the plurality of blocks, which each satisfy AE0xe2x88x92AE greater than xcex1, as possible motion blocks, xcex1 being a threshold value; and a second judging step for judging, when a motion vector of a present block and motion vectors of blocks surrounding the present block have non-random directions and non-random magnitudes, the present block as a motion block, wherein the present block is selected from the possible motion blocks one by one, and wherein the motion block shows a part of an object in motion.
With the above procedure, the first judging step judges a block as a possible motion block when a difference between a first difference sum-total and a second difference sum-total of the block is larger than a threshold value. Accordingly, it is possible to prevent incorrect detection of a block in a flat area (where luminance is even) in the presence of small flicker.
The above object can also be fulfilled by a motion detecting method for detecting, through use of motion vectors outputted from a video encoder which sequentially compression-codes images inputted from a camera, motion of objects in the images, the motion detecting method including: an acquiring step for acquiring a plurality of motion vectors calculated by the video encoder respectively for a plurality of blocks and storing the plurality of motion vectors of the respective plurality of blocks in a memory; a first judging step for detecting motion vectors whose magnitudes are larger than a threshold value among the plurality of motion vectors in the memory and judging blocks corresponding to the detected motion vectors as possible motion blocks; and a second judging step for judging, when a motion vector of a present block and motion vectors of blocks surrounding the present block have non-random directions and non-random magnitudes, the present block as a motion block, wherein the present block is selected from the possible motion blocks one by one, and wherein the motion block shows a part of an object in motion.
With the above procedure, when luminance greatly changes in a fairly short time due to light flash or strong flicker, the mistake of detecting a flat area (such as a wall or a floor where luminance is relatively even) can be avoided.